Model Analysis: Assessing the Dynamics of Student Learning

In this paper we present a method of modeling and analysis that permits the extraction and quantitative display of detailed information about the effects of instruction on a class’s knowledge. The method relies on a cognitive model of thinking and learning that represents student thinking in terms of patterns of association in long-term memory structures that we refer to as schemas or mental models. As shown by previous research, students frequently fail to recognize relevant conditions that lead to appropriate uses of their mental models and, as a result, can use multiple models inconsistently to treat problems that appear equivalent to an expert. Once the most common mental models have been determined via qualitative research, they can be mapped onto probing instruments such as a multiple-choice test. We have developed Model Analysis to analyze the results of these instruments that treats the student as if he/she were in a mixed state – a state which, when probed with a set of scenarios under diverse contextual settings, gives the probability that the student will choose a particular mental model to analyze the scenario. We illustrate the use of our method by analyzing results from the Force Concept Inventory, a research-based multiplechoice instrument developed to probe student’s conceptual understanding of Newtonian Mechanics in a physics class. Model Analysis allows one to use qualitative research results to provide a framework for analyzing and interpreting the meaning of students’ incorrect responses on a well-designed research-based multiple-choice test. These results can then be used to guide instruction, either for an individual teacher or for developers of reform curricula.